1. Field of the Invention
The present invention relates to techniques for copying digital data. More particularly, the invention concerns a copy method supplementing an outboard data copy with a previously instituted copy-on-write logical snapshot to effectively create a duplicate of source data consistent as of a specific time.
2. Description of the Related Art
The advent of fibre channel connectivity used by storage area network (SAN) technology has enabled new paradigms for accessing and copying computer data. The most prevalent copy method uses the SCSI-3 extended copy command (XCOPY) to copy SCSI blocks from one device to another under the control of a copy agent such as a SCSI fibre channel bridge or other SCSI device. The copy agent moves the data outboard from the application host that normally accesses the data.
With outboard copy operations such as the SCSI-3 XCOPY command, data movement operations are off-loaded from the application host by utilizing another process with alternative access to host disks. Some known storage systems 100 (FIG. 1) implement XCOPY functionality using a hardware component in the SAN 102, such as a fibre-channel to SCSI bridge 104. Commands issued to the bridge 104 cause it to directly access blocks from one or more disks 106 attached to the SAN 102 and copy them to another location 108, such as another SAN-attached disk or tape. This copy operation 110 performs input/output (I/O) to host disks 106 outside the path 112 normally used by the file system 116 of the host 114.
When copying large amounts of data, such as file systems or large databases, the copy operation 110 can take some time to complete. If the data being copied is undergoing changes on the host computer 114 while being copied, the copied data will likely not be consistent with any point in time that actually existed on the host 114 because the file system I/O 112 is not coordinated with the outboard copy operation 110.
One known technique to obtain a consistent copy is to make a “physical snapshot” of the source data. In this technique, a mirrored copy (not shown) of the source data is maintained at all times. Then, at the instant when the physical snapshot is desired, the relationship between source and mirrored copy is broken. The mirrored copy thus reflects the content of the source data as of that moment in time. With this approach, an outboard data copy can be created using traditional methods, namely, by copying the (previously mirrored) physical copy. This is possible without disturbing the original data because the physical copy is now disconnected from the source data. Moreover, after creating the physical snapshot, the original file system and physical snapshot can be accessed independently. One disadvantage, however, is that well before the mirrored copy is ever needed, it occupies a significant amount of storage space, namely the same size as the source data.
Another known technique to obtain a consistent copy is the “logical snapshot,” illustrated in FIG. 2A. Unlike a physical snapshot, the logical snapshot works by instantly declaring a body of secondary data 202 to be a duplicate of the source data 204. This may be done by various means, such as adjusting metadata to reflect the declaration, developing content of the secondary data that actually comprises pointers 206 to the source data, or another such technique. The result is a set of secondary data 202 that logically replicates the source data 204, even though the secondary data 202 does not yet physically contain any of the source data 204.
As updates are made to the source data 250 (FIG. 2B), it becomes a mix of original 252 and modified 254 source data. To avoid losing the data replaced by updates, images of each “old” data block must be physically written from the source 250 to a secondary location 256 on an appropriate schedule. Thus, the secondary data 258 becomes a mix of logical data 257 (still unchanged in the source 250) and old blocks 256 physically written from the source 250. Updates can be handled in different ways. In one case, the host continually physically writes “old” data from primary 250 to secondary storage 256, for example in a suitable “background” process. Alternatively, under an “on-demand” technique, the host waits until an update to an item of original (“old”) data is received, and before writing the update to primary 250 the host physically writes the old data item to secondary storage 256. In this way, the secondary copy 258 is still preserved after the point in time at which it was taken. A “copy-on-write” relationship is said to be created between primary 250 and secondary 258 storage.
The physical data 256 is referred to as an “old block file” (OBF) for implementations where data occurs in units of “blocks.” After the logical snapshot has been initiated, when updates to data blocks in the source 250 are received, their original images are copied to the OBF 256 before the updates are physically made to primary storage 250. This only needs to be done once for each distinct block that is updated on the original image. To access the logical snapshot, an entity such as a host volume manager through a device driver filter checks to see if the requested block has been updated. If it has never been updated, the block is represented by the logical secondary data 257, and therefore must be obtained from the original volume 250. Otherwise, if the requested block has been updated one or more times, the OBF 256 is the sole source for obtaining the original block image. The advantage of the logical snapshot approach, then, is that little additional disk space is needed, namely the disk space used to store original blocks, which is therefore dependent on the number of distinct blocks that get updated after the snapshot is taken. This is usually significantly less than twice the original disk space (the amount required by the physical snapshot).
Still, the logical snapshot techniques are not completely perfect. Accordingly, applications that are accessing the logical snapshot 258 need to coordinate their access to blocks in order to be assured of accessing the correct original block images, either in the OBF 256 or the original disk image 252. Conflict can arise between applications accessing the blocks 252 as primary data 250 and those applications accessing the blocks 252 as part of the secondary data 258. If a block is in the process of being copied to the OBF as a result of a source block update, then any applications accessing the secondary copy must wait for the block to be written to the secondary copy before accessing it there. This necessitates some locking or arbitration scheme. For example, a backup application that is copying data from the logical snapshot 258 will have to lock the file(s) 252 being copied so that their blocks are not updated while being copied to the backup media (not shown). This imposes overhead on the application even if the copy operations are being performed outboard from the application server.
In an attempt to address some of these concerns, some have proposed various new copy techniques, including the possible combination of the logical snapshot with outboard data movement. Unfortunately, the use of logical snapshot techniques with outboard data movement introduces complexity. One reason, referring to FIG. 1, is that the logical snapshot is virtualized through a device driver on the original host 114 and is thus not directly accessible by the outboard copy engine 104. Namely, a file system I/O driver virtualizes the snapshot, so that outboard copy engine 104 does not know which blocks to copy in obtaining the logical snapshot image. The typical solution to this involves locking objects (files) on the host 114 so that their blocks cannot be changed, mapping those blocks to their constituent volumes, and then copying the mapped blocks. Although this method successfully obtains the proper blocks from the original volume and OBF, the overhead imposed on the host 114 during the copy operation can compromise the desired goal of performing the copy operation without substantial impact to host applications. IN any event, the locking and mapping activity does impact the application.
A different approach, considered by the present inventors yet unknown in the prior art, caches updated blocks in a new block file during the time that the file system 116 is being updated and the snapshot maintained. This technique, however, has certain risks because failures could leave the file system in a corrupt state or potentially lose all application updates.
Consequently, the existing copy techniques using logical snapshot copy-on-write, outboard data copy, and combinations of these are not completely adequate for all applications due to certain unsolved problems.